Page navigation method and electronic device

ABSTRACT

An electronic apparatus and a page navigation method thereof are provided. The page navigation method includes obtaining first navigation information from a first information source, and obtaining second navigation information from a second information source, generating a first page navigation command based on the first navigation information, and generating a second page navigation command based on the second navigation information, and executing the first page navigation command and the second page navigation command.

CROSS-REFERENCE TO RELATED APPLICATION(S)

This application is based on and claims priority under 35 U.S.C. §119(a) of a Chinese patent application number 202010632544.5, filed onJul. 2, 2020, in the China Intellectual Property Office, and of aChinese patent application number 202011238921.3, filed on Nov. 9, 2020,in the China Intellectual Property Office, the disclosure of each ofwhich is incorporated by reference herein in its entirety.

BACKGROUND 1. Field

The disclosure relates to the field of electronic device. Moreparticularly, the disclosure relates to a page navigation method andelectronic device.

2. Description of Related Art

Page navigation refers to moving a focus, such as a cursor or a focusbox, from one position to another on a page, for example, moving thecursor from one element to another with a mouse.

Generally, a personal computer is equipped with a mouse and a keyboard,and a modern mobile device is equipped with a touch screen, both ofwhich may be used to quickly move the focus. However, at present, thereare many electronic devices without a mouse, a keyboard, and a touchscreen, for example, large-screen devices such as televisions (TVs) andprojection devices that require using arrow keys of a remote control,and special-shaped devices such as virtual reality (VR)/augmentedreality (AR) (VR/AR) devices. None of these have a mouse or can betouched. In addition, for a traditional device, there are cases whereperipherals such as a mouse, a keyboard, and a touch screen are unableto be used due to its physical defects.

Hence, technologies of the related art excessively rely on a mouse, akeyboard, and a remote control, and have the disadvantage of a singlenavigation information source, which is not conducive to navigationefficiency. For example, arrow keys on the keyboard may be used to movethe focus from one element to another. However, the focus must gothrough all elements on the path and can require many keystrokes,thereby reducing the navigation efficiency.

The above information is presented as background information only toassist with an understanding of the disclosure. No determination hasbeen made, and no assertion is made, as to whether any of the abovemight be applicable as prior art with regard to the disclosure.

SUMMARY

Aspects of the disclosure are to address at least the above-mentionedproblems and/or disadvantages and to provide at least the advantagesdescribed below. Accordingly, an aspect of the disclosure is to providea page navigation method, apparatus and electronic device, which canimprove page navigation efficiency.

The technical solution of embodiments of the disclosure is implementedas follows.

Additional aspects will be set forth in part in the description whichfollows and, in part, will be apparent from the description, or may belearned by practice of the presented embodiments.

In accordance with an aspect of the disclosure, a page navigation methodis provided. The page navigation method includes obtaining firstnavigation information from a first information source, and obtainingsecond navigation information from a second information source,generating a first page navigation command based on the first navigationinformation, and generating a second page navigation command based onthe second navigation information, and executing the first pagenavigation command and the second page navigation command.

In an embodiment, the first information source includes at least one ofa mouse, a keyboard and a remote control, and the second informationsource includes at least one of a voice microphone, a two-dimensionalcamera, a three-dimensional camera, an infrared camera, a motion sensor,a posture sensor, a wearable device, a shared sensor information sourceon a remote mobile device, an analog data generator, or an analog remotecontrol.

In an embodiment, the executing of the first page navigation command andthe second page navigation command includes executing the second pagenavigation command to move a focus in a page according to a secondmovement speed, and executing the first page navigation command to movethe focus in the page according to a first movement speed, wherein thesecond movement speed is greater than the first movement speed.

In an embodiment, the generating of the second page navigation commandbased on the second navigation information includes parsing a page todetermine points of interest, and generating the second page navigationcommand for controlling a focus to jump between the points of interest,based on the second navigation information.

In an embodiment, the points of interest include interactive elements ornon-interactive elements.

In an embodiment, the generating of the second page navigation commandbased on the second navigation information includes parsing a tiled pagecontaining a plurality of TAB tables to generate a cascading state tablecontaining the plurality of TAB tables, generating the second pagenavigation command for displaying the cascading state table, and theexecuting of the first page navigation command and the second pagenavigation command includes executing the second page navigation commandto display the cascading state table, and executing the first pagenavigation command to traverse a focus between the TAB tables of thecascading state table.

In an embodiment, the displaying of the cascading state table includesat least one of displaying the cascading state table in a rotatingdisplay mode, displaying the cascading state table in a tiled displaymode, or displaying the cascading state table in a tree display mode.

In an embodiment, the executing of the first page navigation command andthe second page navigation command includes executing the second pagenavigation command to switch a predetermined candidate focus in a pageto an active focus, and executing the first page navigation command tomove a focus in the page from a position of the active focus.

In an embodiment, the page navigation method further includes setting aninitial position of the candidate focus as a position that a userrecently clicked on a page or panel, according to a user click historyon the page or panel, or discretely placing candidate focuses on aplurality of areas of the page or panel according to a number ofcandidate focuses.

In accordance with another aspect of the disclosure, an electronicdevice is provided. The electronic device includes an obtaining moduleconfigured to obtain first navigation information from a firstinformation source, and obtain second navigation information from asecond information source, a command generation module configured togenerate a first page navigation command based on the first navigationinformation, and generate a second page navigation command based on thesecond navigation information, and an execution module configured toexecute the first page navigation command and the second page navigationcommand.

In an embodiment, the first information source includes at least one ofa mouse, a keyboard, or a remote control, and the second informationsource includes at least one of a voice microphone, a two-dimensionalcamera, a three-dimensional camera, an infrared camera, a motion sensor,a posture sensor, a wearable device, a shared sensor information sourceon a remote mobile device, an analog data generator, or an analog remotecontrol.

In an embodiment, the execution module is further configured to executethe second page navigation command to move a focus in a page accordingto a second movement speed, and execute the first page navigationcommand to move the focus in the page according to a first movementspeed, wherein the second movement speed is greater than the firstmovement speed.

In an embodiment, the command generation module is further configured toparse a page to determine points of interest and generate the secondpage navigation command for controlling a focus to jump between thepoints of interest, based on the second navigation information.

In an embodiment, the points of interest include interactive elements ornon-interactive elements.

In an embodiment, the command generation module is further configured toparse a tiled page containing a plurality of TAB tables to generate acascading state table containing the plurality of TAB tables, andgenerate the second page navigation command for displaying the cascadingstate table, and the execution module is further configured to executethe second page navigation command to display the cascading state tableand execute the first page navigation command to traverse a focusbetween the TAB tables of the cascading state table.

In an embodiment, the execution module is further configured to executeat least one of displaying the cascading state table in a rotatingdisplay mode, displaying the cascading state table in a tiled displaymode, or displaying the cascading state table in a tree display mode.

In an embodiment, the execution module is further configured to executethe second page navigation command to switch a predetermined candidatefocus in a page to an active focus, and execute the first pagenavigation command to move a focus in the page from a position of theactive focus.

In an embodiment, the execution module is further configured to set aninitial position of the candidate focus as a position that a userrecently clicked on a page or panel, according to a user click historyon the page or panel, or discretely place candidate focuses on aplurality of areas of the page or panel according to a number ofcandidate focuses.

In accordance with another aspect of the disclosure, an electronicdevice is provided. The electronic device includes a processor and amemory. The memory stores an application program executable by theprocessor, and the application program causes the processor to executeone of the above methods.

In accordance with yet another aspect of the disclosure, acomputer-readable storage medium is provided. The computer-readablestorage medium stores computer-readable instructions, wherein thecomputer-readable instructions are configured to execute one of theabove methods.

It can be seen from the above technical solution that the firstnavigation information is obtained from the first information source andthe second navigation information is obtained from the secondinformation source, the first page navigation command is generated basedon the first navigation information and the second page navigationcommand is generated based on the second navigation information, and thefirst page navigation command and the second page navigation command areexecuted. It can also be seen that in the embodiments of the disclosure,the navigation information is obtained from a plurality of informationsources, and the navigation may be implemented separately based on aplurality of page navigation commands, thereby improving navigationefficiency.

Moreover, the focus may be moved quickly by executing the second pagenavigation command, and the focus may be accurately adjusted byexecuting the first page navigation command.

In addition, a shuttle time may be significantly reduced throughthree-dimensional navigation in a cascading state. Through thecooperation of the first page navigation command and the second pagenavigation command, the focus can be quickly moved based on focusswitching, which can significantly reduce the number of keystrokes.

Other aspects, advantages, and salient features of the disclosure willbecome apparent to those skilled in the art from the following detaileddescription, which, taken in conjunction with the annexed drawings,discloses various embodiments of the disclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other aspects, features, and advantages of certainembodiments of the disclosure will be more apparent from the followingdescription taken in conjunction with the accompanying drawings, inwhich:

FIG. 1 is a flowchart of a page navigation method according to anembodiment of the disclosure;

FIG. 2 is a diagram of driving a focus independently and complementarilyby using an auxiliary navigation layer and other navigation peripheralsaccording to an embodiment of the disclosure;

FIG. 3 is a diagram of quickly navigating a focus to a next element ofinterest according to an embodiment of the disclosure;

FIG. 4 is a diagram of navigation failure caused by function conflictsof arrow keys according to the related art;

FIG. 5 is a diagram illustrating that continuous space navigationshuttles between TABs according to the related art;

FIG. 6 is a diagram of implementing three-dimensional navigation incombination with an auxiliary navigation layer according to anembodiment of the disclosure;

FIG. 7 is a diagram of inputting a word according to the related art;

FIGS. 8A, 8B, 8C, 8D and 8E are diagrams of reducing the number ofkeystrokes in a dense operation area via a multi-input cooperativeoperation according to various embodiments of the disclosure;

FIG. 9 is a diagram of reducing the number of keystrokes required forrepeated page turning operations according to an embodiment of thedisclosure;

FIG. 10 is a diagram of architecture for implementing page navigationaccording to an embodiment of the disclosure;

FIG. 11 is a structural diagram of an electronic device according to anembodiment of the disclosure; and

FIG. 12 is a structural diagram of an electronic device with amemory-processor structure according to an embodiment of the disclosure.

Throughout the drawings, like reference numerals will be understood torefer to like parts, components, and structures.

DETAILED DESCRIPTION

The following description with reference to the accompanying drawings isprovided to assist in a comprehensive understanding to variousembodiments of the disclosure as defined by the claims and theirequivalents. It includes various specific details to assist in thatunderstanding but these are to be regarded as merely exemplary.Accordingly, those of ordinary skill in the art will recognize thatvarious changes and modifications of the various embodiments describedherein can be made without departing from the scope and spirit of thedisclosure. In addition, descriptions of well-known functions andconstructions may be omitted for clarity and conciseness.

The terms and words used in the following description and claims are notlimited to the bibliographical meanings, but, are merely used by theinventor to enable a clear and consistent understanding of thedisclosure. Accordingly, it should be apparent to those skilled in theart that the following description of various embodiments of thedisclosure is provided for illustration purpose only and not for thepurpose of limiting the disclosure as defined by the appended claims andtheir equivalents.

It is to be understood that the singular forms “a,” “an,” and “the”include plural referents unless the context clearly dictates otherwise.Thus, for example, reference to “a component surface” includes referenceto one or more of such surfaces.

For the sake of simplicity and intuition in description, the solutionsof the disclosure will be illustrated by describing several variousembodiments below. A large number of details in the embodiments are onlyused for understanding the solutions of the disclosure. However, it isobvious that the technical solutions of the disclosure may not belimited to these details when implemented. In order to avoidunnecessarily obscuring the solutions of the disclosure, someembodiments are not described in detail, but only a framework isprovided. Hereinafter, “include” means “include but not limited to,” and“according to” means “at least according to . . . , but not limited toonly according to . . . ” When the quantity of components is notspecifically indicated below, it means that the quantity of thecomponents may be one or more, or may be understood as at least one.

In an embodiment of the disclosure, a technical solution for interfacenavigation based on multi-input cooperative operation is provided, whichintegrates input information of other peripherals that may be used fornavigation in addition to existing navigation peripherals (such as amouse and arrow keys), and transforms the information into an auxiliarynavigation input to control the movement of a focus concurrently withthe existing navigation peripherals.

Further, an embodiment of the disclosure also provides a method foraccelerating navigation through three-dimensional accelerated navigationand a dense region candidate focus, which may significantly reduce thenumber of keystrokes and optimize the navigation efficiency.

FIG. 1 is a flowchart of a page navigation method according to anembodiment of the disclosure.

Referring to FIG. 1, at operation 101, first navigation information isobtained from a first information source, and second navigationinformation is obtained from a second information source.

The first information source is preferably a traditional navigationperipheral, such as a mouse, a keyboard, a remote control, and so on.The first navigation information is provided by the first informationsource and used to implement navigation, such as a trigger event ofarrow keys on a keyboard or a remote control, or a click trigger eventof a mouse.

The second information source may be implemented as an auxiliarynavigation peripheral, including at least one of a voice microphone, atwo-dimensional camera, a three-dimensional camera, an infrared camera,a motion sensor, a posture sensor, a wearable device, a shared sensorinformation source on a remote mobile device, an analog data generator,an analog remote control, etc. The second navigation information isprovided by the second information source and used to implementnavigation, such as a gesture instruction detected by thetwo-dimensional camera, or a voice instruction detected by the voicemicrophone.

Various examples of the first information source and the secondinformation source are described. Those skilled in the art shouldunderstand that this description is only exemplary and is not used tolimit the protection scope of the embodiment of the disclosure.

At operation 102, a first page navigation command is generated based onthe first navigation information, and a second page navigation commandis generated based on the second navigation information.

The generating of the first page navigation command based on the firstnavigation information includes: generating a corresponding pagenavigation command based on a trigger event provided by a mouse, akeyboard, a remote control, and the like. For example, navigationcommands of moving a focus up, down, left and right are generated basedon respective trigger events of four arrow keys on the keyboard or theremote control. The generating of the second page navigation commandbased on the second navigation information includes generating acorresponding page navigation command based on sensor informationprovided by the voice microphone, the two-dimensional camera, thethree-dimensional camera, the infrared camera, the motion sensor, theposture sensor, the wearable device, the shared sensor informationsource on the remote mobile device, the analog data generator, or theanalog remote control. For example, a navigation command for moving afocus to the left may be generated based on voice information detectedby the voice microphone, or a navigation command for moving a focus tothe left may be generated based on left-hand sliding informationdetected by the gesture sensor.

At operation 103, the first page navigation command and the second pagenavigation command are executed.

The first page navigation command and the second page navigation commandare independent of each other. The process of executing the first pagenavigation command and the process of executing the second pagenavigation command do not share hardware resources, so the first pagenavigation command and the second page navigation command drive thefocus independently and complementarily.

The meaning of executing the first page navigation command and thesecond page navigation command in operation 102 includes at least one of(1) executing the first page navigation command first, and thenexecuting the second page navigation command, (2) executing the secondpage navigation command first, and then executing the first pagenavigation command, or (3) executing the first page navigation commandand the second page navigation command at the same time.

In one embodiment, in operation 103, the second page navigation commandis executed to move the focus on the page according to a second movementspeed, and the first page navigation command is executed to move thefocus on the page according to a first movement speed. The secondmovement speed is greater than the first movement speed.

It can be seen that the focus may be moved quickly by executing thesecond page navigation command, and the focus may be accurately adjustedby executing the first page navigation command.

In one embodiment, the generating of the second page navigation commandbased on the second navigation information in operation 102 includesparsing the page to determine points of interest, and generating thesecond page navigation command for controlling the focus to jump betweenthe points of interest, based on the second navigation information.Preferably, the points of interest include interactive elements ornon-interactive elements.

Therefore, in an embodiment of the disclosure, the focus may be quicklymoved by executing the second page navigation command for controllingthe focus to jump between the points of interest, and the focus may beaccurately adjusted by executing the first page navigation command.

In an embodiment, the generating of the second page navigation commandbased on the second navigation information in operation 102 includesparsing a tiled page containing a plurality of TAB tables to generate acascading state table containing the plurality of TAB tables, andgenerating the second page navigation command for displaying thecascading state table. The executing of the first page navigationcommand and the second page navigation command in operation 103 includesexecuting the second page navigation command to display the cascadingstate table, and executing the first page navigation command to traversethe focus between the TAB tables of the cascading state table.Preferably, the displaying of the cascading state table includes atleast one of displaying the cascading state table in a rotating displaymode, displaying the cascading state table in a tiled display mode, ordisplaying the cascading state table in a tree display mode.

It can be seen that an embodiment of the disclosure may significantlyreduce the shuttle time through three-dimensional navigation in acascading state.

In an embodiment, the executing of the first page navigation command andthe second page navigation command in operation 103 includes executingthe second page navigation command to switch a predetermined candidatefocus on the page to an active focus, and executing the first pagenavigation command to move the focus on the page from a position of theactive focus.

In one embodiment, according to a user click history on the page orpanel, an initial position of the candidate focus is set as a positionthat the user recently clicked (recently in time) on the page or panel,or according to a number of candidate focuses. The candidate focuses arediscretely placed on a plurality of areas of the page or panel. Thediscrete placing of the candidate focuses on the plurality of areas ofthe page or panel may be implemented as discretely placing the candidatefocuses on the plurality of areas of the page or panel evenly, ordiscretely placing the candidate focuses on the plurality of areas ofthe page or panel unevenly.

Therefore, in an embodiment of the disclosure, through cooperation ofthe first page navigation command and the second page navigationcommand, the focus may be quickly moved based on the focus switching,thereby significantly reducing the number of keystrokes.

Therefore, in an embodiment of the disclosure, a technical solution forin-page navigation based on multi-input cooperative operation isprovided. Under the premise that the existing first information source(such as arrow keys) is consumed by a rendering engine to achievewebpage functions or unchanged space navigation, an auxiliary navigationlayer (corresponding to an auxiliary navigation function achieved basedon the second information source) is newly added in the overall systemperspective. The auxiliary navigation layer integrates input informationof other peripherals that may be used for navigation in addition to theexisting navigation peripherals, and uses this information forcooperative operation and navigation acceleration, so as to achievefunctions that cannot be completed independently by the existingnavigation keys of the related art. Since new and existing navigationtargets both function on and drive focus elements, they may complementeach other. The newly added auxiliary navigation layer is a relativelyindependent subsystem, which does not need to reuse the existingnavigation keys and thus does not conflict with the existing functions.In addition, the auxiliary navigation may be accurate or fuzzy.According to auxiliary signals that the peripherals may provide, anacceleration function may also be provided based on the existingnavigation.

FIG. 2 is a diagram of driving a focus independently and complementarilyby using an auxiliary navigation layer and other navigation peripherals,according to an embodiment of the disclosure.

Referring to FIG. 2, in addition to traditional navigation peripheralscorresponding to the first information source, such as a mouse, akeyboard, and a remote control, an auxiliary navigation layercorresponding to the second information source is further added in anembodiment of the disclosure. On the one hand, the auxiliary navigationlayer collects sensor information available to the system and convertsthe collected sensor information into navigation information. On theother hand, the auxiliary navigation layer plans/optimizes a navigationpath on an abstract layer through the content characteristics of anavigation target, and then executes a navigation operation through aplatform interface to drive the cursor or focus to move to a navigationdestination.

Various embodiments of the disclosure will be described below.

First Embodiment

The first information source may be implemented as a remote control, thesecond information source may be implemented as a posture sensor, and auser simultaneously uses gestures and the remote control to quickly movea focus such as a cursor on a browser.

If a cursor-based navigation drive mode is used on the page, the cursorwill be moved relatively slowly at a given operation by clicking arrowkeys of the remote control. In a process of browsing a webpage or alocal native program page, a large number of clicks (or long presses) ofthe arrow keys may be required to move the cursor.

The auxiliary navigation layer provided by an embodiment of thedisclosure drives the cursor to move by converting the user's gestureinformation into a navigation command. The user may set a frequency ofthe auxiliary navigation layer driving the cursor, to control that thespeed of moving the cursor by gestures is faster than the speed ofmoving the cursor by the arrow keys. For example, the user may quicklymove the cursor through a predetermined gesture of the left hand, and atthe same time slightly adjust a position of the cursor through theremote control in the right hand. Because the two navigation modes donot reuse resources, the gesture navigation may be started or stopped atany time.

Second Embodiment

The first information source may be implemented as a remote control, thesecond information source may be implemented as a gesture sensor, andthe user simultaneously uses gestures and the remote control to quicklymove a focus such as a cursor on a browser.

If the cursor-based navigation drive mode is used on the page, thecursor will be moved relatively slowly at a given operation by clickingarrow keys of the remote control. A large number of clicks (or longpresses) of the arrow keys may be required during the browsing processto drive the cursor to move, because of an important influence factor:normally, the cursor moves continuously and thus, even in a blank area,a plurality of clicks are required to traverse.

FIG. 3 is a diagram of quickly navigating a focus to a next element ofinterest according to an embodiment of the disclosure.

Referring to FIG. 3, the auxiliary navigation layer provided by anembodiment of the disclosure determines interactive elements (such aspictures or videos) as points of interest according to thecharacteristics of the current screen content. In the navigation processbased on gestures, the auxiliary navigation layer skips the blank areaand elements of non-interest on the path, and directly sets the cursorto the next element of interest in the navigation direction. Afterwards,the user may determine to use or not use the arrow keys of the remotecontrol for fine adjustment according to his needs. Since the arrow keysare not affected by the auxiliary navigation layer, the user can alsomove the cursor back to the blank area. Optionally, the user may alsoset the points of interest as all visible elements, and thus only theblank area is skipped when the auxiliary navigation layer moves thecursor.

Third Embodiment

The first information source may be implemented as a keyboard, thesecond information source may be implemented as a microphone, and theuser uses voice to move a focus.

If the running environment of an application uses a navigation drivemode based on space navigation, the focus will be driven to move betweeninteractive elements by clicking arrow keys of the keyboard. On the onehand, due to the large number of Internet pages and defects of webpagecontent authors' design for page accessibility, some webpage componentsmay have already occupied the arrow keys. Accordingly, when the focus ismoved to these elements, the focus cannot be moved away through thearrow keys. For example, drop-down boxes of many websites have thisproblem. On the other hand, some complex components may occupy the arrowkeys even if they are designed in accordance with the recommendations ofthe world wide web consortium (W3C). For example, a playback controlleroften uses up and down keys to control audio volume, and uses left andright keys to control a playback progress bar. Once the focus is movedto such components, the focus cannot be moved away through the arrowkeys.

FIG. 4 is a diagram of navigation failure caused by function conflictsof arrow keys according to the related art.

Referring to FIG. 4, in an embodiment of the disclosure, voicenavigation provided by a microphone may be used as the auxiliarynavigation layer. Since the navigation using the microphone isindependent from the navigation using the arrow keys, the user maydirectly move the stuck focus through other voice commands. Moreover,after the focus is moved away from the stuck position, the user maycontinue to use the four arrow keys of the keyboard to drive the focusto move between the interactive elements.

Fourth Embodiment

The user may implement three-dimensional movement on the page by usingauxiliary navigation.

Regardless of whether the page is web or native, most existing pages areeither based on a personal computer (PC) display environment or based ona mobile device, fewer of them consider a display environment of largescreen and large display area, and fewer of them consider an environmentof an infinite display area (e.g., VR/AR). Due to the limited displayarea, pages are often displayed in categories/pages. For example,referring to FIG. 5, a large number of pages may be classified into aplurality of TAB pages according to different types, and only thecontent of one TAB is displayed currently. The display list of webpagessimilar to search results of a search application is split into aplurality of sub-pages, and only the content of one sub-page isdisplayed currently. Further, function menus at the top of software suchas Office are stacked with different TAB panels, and only one panel isdisplayed at a time. For such pages, in order to navigate from aposition in one TAB page to a position in another TAB page, regardlessof whether the cursor navigation or the focus navigation is used, thecursor or the focus must first be moved to a TAB head and clicked toexpand the corresponding TAB, and then moved to the correspondingposition in the corresponding TAB. Since the moving path of the cursorand the focus is very long, frequent operations may make the userphysically and mentally exhausted.

FIG. 5 is a diagram illustrating that continuous space navigationshuttles between TABs according to the related art.

Referring to FIG. 5, the auxiliary navigation introduced in anembodiment of the disclosure allows the user to use both hands tocontrol. For example, the gesture of the left hand may trigger to entera three-dimensional expansion state, and the folded content of thecurrent page will be expanded in some form (e.g., tree expansion, tiledexpansion, or three-dimensional expansion). Afterwards, the right handdirectly performs penetrating navigation between the expanded TAB pages.After the three-dimensional expansion state is cancelled, the cursor orthe focus has jumped to the corresponding position in the correspondingpage.

For example, the TAB pages that may be folded together may be regardedas a book. Suppose an ant wants to walk from coordinates (e.g., row 10,column 10) of a first page of the book to the coordinates (e.g., row 10,column 10) of a fifth page of the book. In the prior art, the ant mustwalk from the coordinates (row 10, column 10) of the first page to theedge of the first page, then walk from the edge of the first page to theedge of the fifth page, and finally walk from the edge of the fifth pageto the coordinates (row 10, column 10) of the fifth page.

FIG. 6 is a diagram of implementing three-dimensional navigation incombination with an auxiliary navigation layer according to anembodiment of the disclosure.

Referring to FIG. 6, after a three-dimensional expansion mode isentered, movement is equivalent to drilling a hole through the entirebook at a position of the coordinates (e.g., row 10, column 10), and theant may directly walk from the coordinates (row 10, column 10) of thefirst page to the same coordinates (row 10, column 10) of the fifth pagealong the hole with 4 operations. In addition, after thethree-dimensional expansion state is exited, the hole is restored, thebook is opened to the fifth page, and the ant is on the coordinates (row10, column 10) of the fifth page.

Therefore, the three-dimensional focus movement provided by anembodiment of the disclosure can reduce the number of keystrokes forremote control navigations, and the expansion mode may use theadvantages of large screens, large display areas, and the infinitedisplay areas of VR/AR devices.

Fifth Embodiment

The user may use auxiliary navigation to reduce the number of movementsin a dense interaction area.

The input method interface is a typical dense interaction area, whichmay be a native component or a part of webpage content. A beginningtypist knows that each finger must cover a small area, to improve thetyping speed by reducing the movement distance of fingers. Even on amobile phone, it is found that the input with two fingers is much fasterthan that with one finger. However, when the user uses the remotecontrol, it is equivalent to an operation with only one finger. What isworse is that the finger cannot jump yet, and must slide through everykey between two points against the input method interface.

FIG. 7 is a diagram of inputting a word according to the related art.

Referring to FIG. 7, when the focus starts at the letter “D,” in orderto input the word “DOG,” the user must press the enter key 3 times andthe arrow keys 12 times on the remote control, i.e., a total of 15 keyoperations, to complete the input, which can very cumbersome.

The method of combining auxiliary navigation in an embodiment of thedisclosure allows the user to operate with both hands (i.e., withmultiple inputs) and jump forward. An embodiment of the disclosureintroduces candidate focuses and an active focus in the denseinteraction area through the method of multi-input cooperativeoperation. The user may set two or more candidate focuses, and only oneof the candidate focuses may be used as the active focus at any time.The user triggers the switching of the active focus between thecandidate focuses through an additional input mode. After switching, thearrow keys are used to control the movement of the current active focusto achieve jumping.

FIGS. 8A to 8E are diagrams of reducing the number of keystrokes in adense operation area via a multi-input cooperative operation accordingto various embodiments of the disclosure. Two candidate focuses, namelythe keys D and L, are preset.

Referring to FIGS. 8A to 8E, a typical scenario is as follows. The usertriggers the switching of the candidate focuses through a gestureinstruction issued by one hand, and the other hand drives the currentactive focus via the arrow keys after the switching to continuenavigation. In this way, when the focus starts at the letter “D,” “DOG”may be inputted by pressing the enter key 3 times, the arrow keys 2times, and the candidate focus switching operation 2 times. Accordingly,the number of keystrokes is significantly reduced, and the left andright hands can operate cooperatively.

Further, the candidate focus switching operation may also be usedinstead of pressing the enter key to trigger input, thereby furtherreducing the number of keystrokes. For example, the candidate focusswitching operation after the active focus is moved is set to triggerthe direct input of the element under the current active focus. In thisway, the enter key presses for “D” and “O” may be omitted, and the totalnumber of keystrokes may be reduced by almost half.

Further, a page turning scenario may be optimized in combination withthe fourth embodiment.

FIGS. 8A to 8E, an embodiment in which ‘focus’ is a ‘focus box’ isillustrated, but the present disclosure is not limited thereto. That is,instead of the ‘focus box’ shown in 8A to 8E, an ‘active cursor’ and aplurality of ‘candidate cursors’ may be displayed.

FIG. 9 is a diagram of reducing the number of keystrokes required forrepeated page turning operations according to an embodiment of thedisclosure.

Referring to FIG. 9, under normal circumstances, when symbols andletters are inputted, the focus must be moved to a page switching keyfor page turning. By using the optimization method of the fourthembodiment, the number of keystrokes required for a route to and from apage turning key may be avoided. The sensor signal for triggering theswitching between candidate focuses is not limited to a gesture. Thesensor signal may be customized according to different devices and useenvironments. For example, for very close operating environments such asVR and AR, the switching between candidate focuses may be triggered bydetecting eye movement, and the nearest candidate focus in a vectordirection may be found along the movement vector of the user's eye andmay be switched into the current active focus. For another example, aspecial-shaped device provided to a special crowd may be just a simpleswitch.

The embodiments of the disclosure are described above. Those skilled inthe art should understand that this description is only exemplary and isnot used to limit the protection scope of the embodiments of thedisclosure.

It can be seen that the disclosure relates to the navigation control ofa focus and the like during a user's electronic browsing process. Theauxiliary navigation layer (or, referred to as a “global auxiliarynavigation subsystem”) is newly added based on existing traditionalperipherals of the current system, which includes sub-modules such as auser configuration sub-module, a sensor information integrationsub-module, a navigation path planning sub-module, and a navigationcommand execution sub-module. The sensor information integrationsub-module is configured to integrate all sensor and simulationresources available in the current system and convert them intoformatted auxiliary navigation input data. The navigation path planningsub-module performs auxiliary navigation path planning through thecharacteristic analysis results of the currently presented content andthe navigation input data, and obtains a navigation command for output.The navigation command execution sub-module triggers the execution ofnavigation through a suitable system interface.

The auxiliary navigation control provided by the disclosure isindependent from and complementary to the current existing navigationcontrol, and they are independent of each other and do not interferewith each other, thereby providing the user with more flexiblenavigation options. Because the auxiliary navigation control does notshare hardware resources with the existing navigation control, the usercan realize the auxiliary navigation control without reusing the currentfunction keys. The auxiliary navigation control can be enabled andexited in a simple and smooth mode, and may be called at will by theuser. Meanwhile, the newly added auxiliary navigation provides improvedperformance according to the obtained navigation data, and the user mayuse the auxiliary navigation in combination with the existing navigationresources according to the accuracy of the auxiliary navigation.

FIG. 10 is a diagram of architecture for implementing page navigationaccording to an embodiment of the disclosure.

Referring to FIG. 10, the user configures and enables the auxiliarynavigation subsystem, selects an available sensor informationsource/simulation information source, and completes adaptive debuggingof sensor information. The sensor information integration sub-module isenabled. The background process monitors and obtains the input of thesensor/simulation information source, and converts the input intonavigation guidance data for output. After the navigation path planningsub-module receives the input of navigation data, a sampling systempresents the content, obtains positions of navigable elements, analyzesthe navigation path to obtain a destination navigation position, andoutputs a navigation instruction. The navigation command executionmodule calls a corresponding platform system interface according to theincoming navigation instruction, triggers the cursor and the focus tomove, and further triggers navigation operations such as page expansionor jump.

Specifically, the main functions of the user configuration moduleinclude the following:

(1) The user decides whether to enable the auxiliary navigationsubsystem.

(2) All available sensor resources and simulation resources areenumerated. The sensor resources include, but are not limited to:various sensor peripheral resources equipped on the current device,e.g., a voice microphone, various (two-dimensional, three-dimensional,and infrared) cameras, a motion sensor, a posture sensor, etc. Inaddition, the sensor resources also include external sensor informationresources obtained through a sensor information sharing subsystem, e.g.,sensor information on wearable devices such as smart watches obtainedthrough Bluetooth pairing, and sensor information shared on remotemobile devices obtained through unified account association. Thesimulation resources include, but are not limited to: sensor datasimulated by software on the current system or other connected systems,e.g., a variety of sensor information simulated by a software datagenerator, navigation key information sent by a mobile phone simulatedremote control, and so on.

(3) The user debugs the selected information source to further select anappropriate sampling frequency (corresponding to the navigation speedand sensitivity). For an information source with a large number ofcontinuous inputs, data frames may be discarded according to thesampling frequency to adjust the sensitivity. Optionally, a defaultmovement step length may be set, that is, the movement distance of onenavigation operation triggered by simple direction information in theabsence of elements of interest. For example, when the input informationof the sensor source does not include a movement distance, but includesonly several specific simple state changes (corresponding to a specificdirection), it is similar to a default step length obtained throughpressing the arrow keys of the remote control by the user to drive thecursor to move. When no model of interest is configured, the user setsthe default step length of auxiliary navigation to a relatively largevalue, so that a large distance may be jumped according to eachauxiliary navigation command, and then a small distance may be jumped bycooperating with the arrow keys of the remote control, thereby achievingthe cooperative effect of quick adjustment and fine adjustment (e.g., asin the first embodiment).

(4) A model of interest for auxiliary navigation is selected andaccelerated three-dimensional navigation is enabled. The model ofinterest for navigation includes whether to set elements of interest,for example, not set elements of interest, set as interactive elements,set as visible elements, set as a certain type or several types ofspecific elements, set as elements with certain characteristics, etc.The accelerated three-dimensional navigation is used to simplifynavigation steps between folded contents, and the user may set the levelof the accelerated three-dimensional navigation.

(5) The settings are saved and the functions are enabled.

The main functions of the sensor information integration module includethe following:

(1) Available sensor data sources and simulation data sources areenumerated according to the request of the user configuration module.The cross-system sensor information sharing mechanism and systemsimulation belong to external module technologies, and will not bedescribed in the disclosure.

(2) The data source is monitored, and the data is adapted and convertedinto a unified navigation data output format. When the input of the datasource is a simple state, for example, a static direction sign or staticgesture identified by a camera, a simple key input and an analog keyinput, the output is a corresponding direction (a direction type, adirection value). It means to perform one navigation operation in thisdirection to move one step length or move to the next element ofinterest in this direction. When the input source information containsspeed change data, such as a movement track of a mouse, a camera orother sensor identification bodies, the input contains step lengthinformation, and the output is a vector type, a direction value, and amoving distance. When the input source information is a specificinstruction, such as an instruction to jump to a certain element orsearch for elements of keywords, and when the input source informationis a specific gesture or voice and a specified function triggered by afunction key, such as a function of directly jumping back to a header orexpanding a folded page, the output is an instruction type, aninstruction value, and an instruction parameter.

(3) The converted data is queued and is sent to the navigation pathplanning module at a preset rate, or the data is discarded.

(4) The system state is monitored, and when the change of the datasource or a new data source is discovered, the user is notified toadjust the configuration.

The navigation path planning module generates a navigation command fordescribing a destination required for this navigation after receivingthe input of the sensor information integration data module.

The navigation command execution module determines whether the currentnavigation is cursor navigation or focus navigation.

(1) If it is the cursor navigation, the navigation command executionmodule obtains the current position of the cursor, and then processesthe input of the sensor information integration data module.

If an input of direction type is received and no model of interest isconfigured, a target position is calculated according to the currentposition of the cursor and the default step length. If the targetposition exceeds the boundary of a viewport, the target position istruncated at the boundary of the viewport, and the position is sent tothe navigation command execution module.

If the input of direction type is received and a model of interest isconfigured, then a node tree corresponding to the model of interest isfirst obtained from a content analysis module. The node tree istraversed by taking the current point as a starting point, and a nearestnode is searched for in the target direction according to a spacenavigation algorithm. If the nearest node is found, the target positionis calculated through the target node; if any node is found in thetarget direction, the target position is calculated according to thecurrent position of the cursor and the default step length. If thetarget position exceeds the boundary of the viewport, the targetposition is truncated at the boundary of the viewport, and the positionis sent to the navigation command execution module.

If an input of vector type is received, regardless of whether a model ofinterest is configured, a target position is calculated according to thecurrent position of the cursor and the vector information. If the targetposition exceeds the boundary of the viewport, the target position istruncated at the boundary of the viewport, and the position is sent tothe navigation command execution module.

If an input of instruction type is received, an instruction list isqueried, and the calculation related to the corresponding instruction istriggered.

(2) If it is the focus navigation, the content analysis sub-moduleobtains a position of the current focus element, and then processes theinput of the sensor information integration data module.

If an input of direction type is received and no model of interest isconfigured, the points of interest are set as interactive elements, andthen the next operation is performed.

If the input of direction type is received and a model of interest isconfigured, then a node tree corresponding to the model of interest isfirst obtained from the content analysis module. The node tree istraversed by taking the current focus element as a starting point, and anearest node is searched for in the target direction according to aspace navigation algorithm. If the nearest node is found, the targetnode is sent to the navigation command execution module. If any node isnot found in the target direction, the current focus cannot be moved.

If an input of vector type is received, the vector information isignored, and the processing is the same as that used for the input ofdirection type.

If an input of instruction type is received, an instruction list isqueried, and the calculation related to the corresponding instruction istriggered.

At present, almost all operating system platforms support obtaining andsetting a position of the cursor, and also provide accessibility systeminterfaces for content (e.g., Microsoft Active Accessibility/InterfaceAccessible (MSAA/IAccessible), UIAExpress and IAccessible2 of Windows,NSAccessibility of Mac OS X, assistive technology-service providerinterface (AT-SPI) of Linux, Accessibility framework of Android,UIAccessibility of iOS, etc.). A navigation command based on cursornavigation may be executed through the system interface set for thecursor, and a focus may be reversely set for the content element throughthe accessibility system interface, thereby executing the focus-basednavigation command.

The main functions of the navigation command execution module includethe following:

(1) A platform top window is obtained as a target context.

(2) It is queried whether an interface for the cursor is set in thetarget context. If yes, it is indicated that the navigation is based onthe cursor, otherwise, it is indicated that the navigation is based onthe focus.

(3) The accessibility interface provided by the platform system isregistered, changes in the content of a top-layer application arequeried and responded, an accessible node tree is constructed, andinteraction with the node tree is prepared.

(4) After the input of the navigation path planning module is received,if the navigation is based on the cursor, the interface set for thecursor by the platform is called, and the cursor is set to the targetposition. If the navigation is based on the focus, a corresponding nodeon the accessible node tree is found by matching the incoming element,and a focus setting event is sent through the node. The focus settingevent notifies an application framework through the accessible interfaceof the platform of completing the operation of setting the focus, andthe application framework reports the focus setting event through ajavascript (JS) engine, thereby achieving the synchronization ofhigher-layer APP logics.

The main functions of the content analysis sub-module include thefollowing:

(1) First, based on the accessible node tree constructed by thenavigation command execution module, a target type content node subtreeis constructed through filtering the model of interest, and is updatedsynchronously with the accessible node tree at any time.

(2) When the accessible node tree is incomplete and is difficult toobtain, the screen display content is periodically sampled through anexternal video and image analysis system (for example, a characteristicanalysis system in FIG. 10), and an object of the screen display contentis analyzed and extracted to assist in construction of a content nodetree.

(3) A traversal interface for the content node tree is provided toprovide node information for searching for a next focus according to thespace navigation algorithm.

Function (2) is an optional function.

For the execution of special navigation instructions, the system maypreset the special navigation instructions, and the user associates anyexternal trigger mechanism such as voice or gestures with these presetnavigation instructions to accelerate the movement of the focus orcursor.

When the navigation path planning module receives the special navigationinstructions, an instruction flow is as follows:

(1) An instruction of returning to a header.

Operation 1: the accessible node tree is traversed in the root directionfrom the current focus element, to find a first node whose role is aheader type as a target node. Operation 2: if the current context iscursor-based navigation, a position of the node is queried, and theposition is sent to the navigation command execution module. Operation3: if the current context is focus-based navigation, the target node isdirectly sent to the navigation command execution module.

(2) An instruction of expanding table elements at the same level andperforming fast three-dimensional navigation.

Operation 1: the current position of the focus is queried, theidentifier of a header node of the classification panel where thecurrent position of the focus is located is queried, the currentposition of the focus and the identifier of the header node of theclassification panel are cached in a stack (TAB ID, START_POSITION), anda thumbnail of the classification panel is cut and cached(TAB_ID_SNAPSHOT). Operation 2: header nodes of all other classificationpanels at the same level as the current classification panel aretraversed one by one, a focus is set for a header node, a click orconfirmation event is sent to trigger the display of the classificationpanel, and a thumbnail of the classification panel is cut and cached. Ifthe user sends a confirmation message at this time, it is indicated thatthe user desires to navigate to the current panel, and then thetraversal of the panels is stopped. When the traversal of classificationpanels at the same level is completed but the user has not sent aconfirmation signal, operation 3 is performed. Operation 3: thethumbnails of the classification panels are displayed in a cascading androtating manner, until the user sends a signal to confirm the selectionof a certain classification panel or an exit signal. For the selectionconfirmation signal, the user is navigated to a table corresponding tothe current thumbnail, the focus is set to an element corresponding tothe START_POSITION of the classification panel, and then thethree-dimensional navigation mode is exited. For the exit signal, thethree-dimensional navigation mode is exited, and the focus is set backto the corresponding position of the classification table when thethree-dimensional navigation mode is entered. Operation 4: after thethree-dimensional navigation is performed on the same level ofclassification panels for the first time, the header nodes andcorresponding thumbnails of the classification panels have been cached.Afterwards, when a three-dimensional navigation command request isreceived again, the cached thumbnails of the classification panels aredirectly displayed in a cascading and rotating manner for the user tochoose, which is the same as operation 3.

In addition, the optimization process for the dense operation area (suchas but not limited to the input method panel) in an embodiment of thedisclosure includes the following:

Operation 1: when the user enables this optimization method or detectsthat a dense panel appears at the top-layer of an application, marks ofcandidate focus are first drawn on the screen. The drawing may includebut is not limited to: the candidate focuses are represented by dottedboxes or dotted cursors, and the current active focus is represented bya solid box. One candidate focus or a plurality of candidate focuses maybe specified according to the presetting or user configuration. A methodfor selecting an initial position of the candidate focus includes: (1)According to a user's click history on the page or panel, the initialposition of the candidate focus is set as a position that the userrecently clicked on the page or panel; or (2) according to a number ofcandidate focuses, the candidate focuses are discretely (evenly orunevenly) placed on a plurality of areas of the page or panel. An arrayof candidate focuses is initialized, and state information of eachcandidate focus is recorded, such as identity (ID), whether it isactive, or a position (and its corresponding accessible node).

Operation 2: the movement of the actual focus element on the panel ofthe application is monitored through the accessible node tree of thenavigation command execution module, the position of the current activefocus (and its corresponding accessible node information) is updated,and the drawing of the navigation mark (solid box) is updated. Operation3: when a candidate focus switching command is received, a targetcandidate focus is queried in the array of candidate focuses and is setas a new active focus. If the current active focus has moved before, thenavigation command execution module sends a confirmation signal to thecurrent element to trigger the input. The state of the current activefocus is set to be inactive, and the mark drawn on the panel is updatedas the mark of a candidate focus (e.g., a dotted box). The state of thenew candidate focus is updated to be active, and the mark of the newcandidate focus drawn on the panel is updated as the mark of the activefocus (e.g., a solid box). The navigation command execution module setsthe node element corresponding to the new active focus element as afocus element for the application.

Based on the above description, an embodiment of the disclosure furtherprovides an electronic device.

FIG. 11 is a structural diagram of a page navigation device according toan embodiment of the disclosure.

Referring to FIG. 11, the electronic device includes: an obtainingmodule 1101, configured to obtain first navigation information from afirst information source, and obtain second navigation information froma second information source, a command generation module 1102,configured to generate a first page navigation command based on thefirst navigation information, and generate a second page navigationcommand based on the second navigation information, and an executionmodule 1103, configured to execute the first page navigation command andthe second page navigation command.

In an embodiment, the first information source includes at least one ofa mouse, a keyboard, or a remote control, and the second informationsource includes at least one of a voice microphone, a two-dimensionalcamera, a three-dimensional camera, an infrared camera, a motion sensor,a posture sensor, a wearable device, a shared sensor information sourceon a remote mobile device, an analog data generator, or an analog remotecontrol.

In an embodiment, the execution module 1103 is further configured toexecute the second page navigation command to move a focus on a pageaccording to a second movement speed, and execute the first pagenavigation command to move the focus on the page according to a firstmovement speed. The second movement speed is greater than the firstmovement speed.

In an embodiment, the command generation module 1102 is furtherconfigured to parse the page to determine points of interest, andgenerate the second page navigation command for controlling the focus tojump between the points of interest, based on the second navigationinformation. Preferably, the points of interest include interactiveelements or non-interactive elements.

In an embodiment, the command generation module 1102 is furtherconfigured to parse a tiled page containing a plurality of TAB tables togenerate a cascading state table containing the plurality of TAB tables,and generate the second page navigation command for displaying thecascading state table. The execution module 1103 is further configuredto execute the second page navigation command to display the cascadingstate table, and execute the first page navigation command to traversethe focus between the TAB tables of the cascading state table.

In an embodiment, the execution module 1103 is further configured toexecute at least one of displaying the cascading state table in arotating display mode, displaying the cascading state table in a tileddisplay mode, or displaying the cascading state table in a tree displaymode.

In an embodiment, the execution module 1103 is further configured toexecute the second page navigation command to switch a predeterminedcandidate focus on the page to an active focus, and execute the firstpage navigation command to move the focus on the page from a position ofthe active focus.

In an embodiment, the execution module 1103 is further configured toset, according to a user's click history on the page or panel, theinitial position of the candidate focus as a position that the userrecently clicked on the page or panel, or, discretely place, accordingto the number of candidate focuses, the candidate focuses on a pluralityof areas of the page or panel. The process of discretely placing thecandidate focuses on the plurality of areas of the page or panel may beimplemented as discretely placing the candidate focuses on the pluralityof areas of the page or panel evenly, or discretely placing thecandidate focuses on the plurality of areas of the page or panelunevenly.

Based on the above, embodiments of the disclosure provide an in-pagenavigation solution for triggering a cursor or focus to move through thecooperative operation of multi-input devices. A global auxiliarynavigation layer is introduced to separate the input of the existingnavigation signals from the input of other navigation signals. Theauxiliary navigation signals use an independent navigation rule, so thatresources do not conflict and a running basis for the cooperativeoperation of multiple devices may be provided. Moreover, by using theindependent rule, the auxiliary navigation layer may be smoothly enabledand exited, as described in the fourth embodiment and the fifthembodiment. The user does not need to interrupt the current navigationto enable the new function, and the new navigation function is almostparallel to the existing navigation function. Further, the navigationrule of the auxiliary navigation layer may be complicated or simple, anda purpose is to customize operations that cannot be done independentlyby the existing navigation modes. Furthermore, the embodiments of thedisclosure provide three-dimensional accelerated navigation based on theexpansion of closed panels in a page and accelerated navigation based oncandidate focus switching for a dense operation area. In addition, thethree-dimensional accelerated navigation based on the expansion ofclosed panels uses a space-for-time method, which provides user choicesby caching the expanded content to reduce the user's click operations.The accelerated navigation based on candidate focus switching for adense operation area is described by taking an input method panel as anexample. In order to increase the parallelism of operations, a newsignal is used to trigger the focus to jump, which reduces keyoperations on the frequently repeated operation path. A difficulty inthe implementation of the global auxiliary navigation layer is tointeract with the currently presented content through a compatiblemethod to move the focus.

An embodiment of the disclosure further provides an electronic devicewith a memory-processor structure.

FIG. 12 is a structural diagram of an electronic device with amemory-processor structure according to the disclosure.

Referring to FIG. 12, an electronic device with a memory-processorstructure includes a processor 1201 and a memory 1202. The memory 1202stores an application program executable by the processor 1201, whichcauses the processor 1201 to execute the above methods for theelectronic device.

The memory 1202 may be specifically implemented as various storage mediasuch as an electrically erasable programmable read-only memory (EEPROM),a flash memory, or a programmable read-only memory (PROM). The processor1201 may be implemented to include one or more central processing unitsor one or more field programmable gate arrays (FPGAs). The fieldprogrammable gate array integrates one or more central processing unitcores. Specifically, the central processing unit or central processingunit core may be implemented as a central processing unit (CPU) ormicrocontroller unit (MCU).

In an embodiment according to the disclosure, the processor isconfigured to obtain first navigation information from a firstinformation source and obtain second navigation information from asecond information source, generate a first page navigation commandbased on the first navigation information and generate a second pagenavigation command based on the second navigation information, andexecute the first page navigation command and the second page navigationcommand.

The first information source comprises at least one of a mouse, akeyboard, or a remote control, and the second information sourcecomprises at least one of a voice microphone, a two-dimensional camera,a three-dimensional camera, an infrared camera, a motion sensor; aposture sensor, a wearable device, a shared sensor information source ona remote mobile device, an analog data generator, or an analog remotecontrol.

In an embodiment according to the disclosure, the processor isconfigured to execute the second page navigation command to move a focusin a page according to a second movement speed, and execute the firstpage navigation command to move the focus in the page according to afirst movement speed. The second movement speed is greater than thefirst movement speed.

In an embodiment according to the disclosure, the processor isconfigured to parse a page to determine points of interest, and generatethe second page navigation command for controlling a focus to jumpbetween the points of interest, based on the second navigationinformation.

The points of interest comprise interactive elements or non-interactiveelements.

In an embodiment according to the disclosure, the processor isconfigured to parse a tiled page containing a plurality of TAB tables togenerate a cascading state table containing the plurality of TAB tables,and generate the second page navigation command for displaying thecascading state table. The processor is further configured to executethe second page navigation command to display the cascading state table,and execute the first page navigation command to traverse a focusbetween the TAB tables of the cascading state table.

In an embodiment according to the disclosure, the processor isconfigured to display the cascading state table in at least one displaymode among a rotating display mode, a tiled display mode, or a treedisplay mode.

In an embodiment according to the disclosure, the processor isconfigured to execute the second page navigation command to switch apredetermined candidate focus in a page to an active focus, and executethe first page navigation command to move a focus in the page from aposition of the active focus.

In an embodiment according to the disclosure, the processor isconfigured to set an initial position of the candidate focus as aposition that a user recently clicked on a page or panel, according to auser's click history on the page or panel, or discretely place candidatefocuses on a plurality of areas of the page or panel according to anumber of candidate focuses.

In the embodiments of the disclosure, the first navigation informationis obtained from the first information source and the second navigationinformation is obtained from the second information source, the firstpage navigation command is generated based on the first navigationinformation and the second page navigation command is generated based onthe second navigation information, and the first page navigation commandand the second page navigation command are executed. It can be seen thatin the embodiments of the disclosure, the navigation information isobtained from a plurality of information sources, and the navigation maybe implemented separately based on a plurality of page navigationcommands, thereby improving navigation efficiency.

Moreover, the focus may be moved quickly by executing the second pagenavigation command, and the focus may be accurately adjusted byexecuting the first page navigation command.

In addition, the shuttle time may be significantly reduced throughthree-dimensional navigation in a cascading state. Through thecooperation of the first page navigation command and the second pagenavigation command, the focus may be quickly moved based on focusswitching, which may significantly reduce the number of keystrokes.

According to another embodiment, the processor 1201 may control thedisplay (not illustrated) to display a plurality of candidate focuses onthe screen, switch one of the plurality of candidate focuses to anactive focus according to a first user input, and control the display tomove the switched active focus according to a second user input. Here,the focus may be one of a cursor or a focus box.

In this case, the first user input may be received through a firstinformation source, and the second user input may be received through asecond information source. For example, the first information sourcecomprises at least one of a voice microphone, a two-dimensional camera,a three-dimensional camera, an infrared camera, a motion sensor, aposture sensor, a wearable device, a shared sensor information source ona remote mobile device, an analog data generator, or an analog remotecontrol. The second information source comprises at least one of amouse, a keyboard, or a remote control. In particular, the processor mayset an initial position of the predetermined candidate focus as aposition that a user recently clicked on the screen, according to auser's click history on the screen.

In addition, the processor is configured such that the plurality ofcandidate focuses are discretely placed on a plurality of areas of thescreen according to a number of candidate focuses.

In this case, the screen may include a keyboard UI including UI elementscorresponding to a plurality of characters, and the plurality ofcandidate focuses may be displayed on elements corresponding to at leasttwo or more characters from among the plurality of characters. Forexample, as illustrated in FIG. 8A, the plurality of candidate focusesmay be displayed on elements corresponding to the characters on thekeyboard. In addition, when one of the plurality of candidate focuses isswitched to an active focus, the processor 1201 may control the displayto display the switched active focus distinctively from the candidatefocuses.

In this case, the screen may include a keyboard UI including UI elementscorresponding to a plurality of characters, and the plurality ofcandidate focuses may be displayed on elements corresponding to at leasttwo or more characters from among the plurality of characters. Forexample, as illustrated in FIG. 8A, the plurality of candidate focusesmay be displayed on elements corresponding to the characters on thekeyboard.

In addition, when one of the plurality of candidate focuses is switchedto an active focus, the processor 1201 may control the display todisplay the switched active focus distinctively from the candidatefocuses.

It should be noted that not all operations and modules in the aboveprocesses and structural diagrams are necessary, and some operations ormodules may be omitted according to actual requirements. The executionsequence of each operation is not constant, but may be adjusted asrequired. The division of each module is only for the convenience ofdescribing the used functions. During actual implementation, one modulemay be implemented as a plurality of modules, and the functions of aplurality of modules may also be implemented by the same module. Thesemodules may be located in the same device or in different devices.

The hardware modules in each embodiment may be implemented with amechanical or electronic manner. For example, a hardware module mayinclude specially designed permanent circuits or logic devices (forexample, dedicated processors, such as FPGAs or application-specificintegrated circuits (ASICs)) to complete a specific operation. Thehardware module may also include programmable logic devices or circuitstemporarily configured by software (for example, general-purposeprocessors or other programmable processors) to perform a specificoperation. It may be determined according to costs and timeconsiderations whether to use the mechanical manner, or the dedicatedpermanent circuit, or the temporarily configured circuit (for example,configured by software) to implement the hardware module.

The disclosure further provides a machine-readable storage medium, whichstores instructions for causing a machine to perform the method of theapplication. Specifically, a system or device equipped with a storagemedium may be provided. The storage medium stores software program codesfor realizing the functions of any one of the above embodiments, and acomputer (or CPU or MPU) of the system or the device may read andexecute the program codes stored in the storage medium. In addition,part or all of the actual operations may be completed by an operatingsystem and the like running on the computer through the instructionsbased on the program codes. The program codes read from the storagemedium may also be written into a memory configured in an expansionboard inserted into the computer or written into a memory configured inan expansion unit connected to the computer, and then part or all of theactual operations are executed by a CPU and the like installed on theexpansion board or the expansion unit through the instructions based onthe program codes, thereby realizing the functions of any one of theabove embodiments.

In embodiments, the storage medium for providing the program codes mayinclude a floppy disk, a hard disk, a magneto-optical disk, an opticaldisk (e.g., compact disc read only memory (CD-ROM), compactdisc—recordable (CD-R), compact disc-rewritable (CD-RW), digitalversatile disc-read only memory (DVD-ROM), digital versatile disc—randomaccess memory (DVD-RAM), digital versatile disc-rewritable (DVD-RW orDVD+RW)), a magnetic tape, a non-transitory memory card and an ROM.Optionally, the program codes may be downloaded from a server computeror a cloud via a communication network.

The term “exemplary” means “serving as an example, instance ordescription,” and illustrations and embodiments described as “exemplary”should not be interpreted as a more preferred or more advantageoustechnical solution. For the purpose of conciseness of drawings, eachfigure only schematically shows the part related to the disclosure, butdoes not represent the actual structure of a product. Moreover, in orderto make the drawings concise and easy to understand, in some drawings,only one of components with the same structure or function isschematically shown, or only one of them is marked. Herein, “one” doesnot mean that the number of relevant parts of the disclosure is limitedto “only one,” and “one” also does not mean to exclude the situationwhere the number of relevant parts of the disclosure is “more than one.”Herein, “upper,” “lower,” “front,” “rear,” “left,” “right,” “inner,”“outer,” etc., are only used to indicate the relative positionrelationship between relevant parts, but are not used to limit theabsolute positions of the relevant parts.

While the disclosure has been shown and described with reference tovarious embodiments thereof, it will be understood by those skilled inthe art that various changes in form and details may be made thereinwithout departing from the spirit and scope of the disclosure as definedby the appended claims and their equivalents.

What is claimed is:
 1. A page navigation method, comprising: displayinga plurality of candidate focuses on a screen; switching one of theplurality of candidate focuses to an active focus according to a firstuser input; and moving the switched active focus according to a seconduser input.
 2. The method of claim 1, wherein the first user input isreceived through a first information source, and wherein the second userinput is received through a second information source.
 3. The method ofclaim 2, wherein the first information source comprises at least one ofa voice microphone, a two-dimensional camera, a three-dimensionalcamera, an infrared camera, a motion sensor, a posture sensor, awearable device, a shared sensor information source on a remote mobiledevice, an analog data generator, or an analog remote control, andwherein the second information source comprises at least one of a mouse,a keyboard, or a remote control.
 4. The method of claim 1, furthercomprising: setting an initial position of a predetermined candidatefocus as a position that a user recently clicked on the screen,according to a user's click history on the screen.
 5. The method ofclaim 4, wherein the plurality of candidate focuses are discretelyplaced on a plurality of areas of the screen according to a number ofcandidate focuses.
 6. The method of claim 1, wherein the screen includesa keyboard user interface (UI) including UI elements corresponding to aplurality of characters.
 7. The method of claim 6, wherein the pluralityof candidate focuses are displayed on elements corresponding to two ormore characters from among the plurality of characters.
 8. The method ofclaim 1, wherein the focus is one of a cursor or a focus box.
 9. Themethod of claim 1, wherein the switching comprises, based on one of theplurality of candidate focuses being switched to the active focus,displaying the switched active focus distinctively from candidatefocuses.
 10. An electronic device, comprising: a display; a memory; anda processor configured to execute at least one instruction stored in thememory to control the electronic device, wherein the processor isfurther configured to: control the display to display a plurality ofcandidate focuses on a screen, switch one of the plurality of candidatefocuses to an active focus according to a first user input, and controlthe display to move the switched active focus according to a second userinput.
 11. The electronic device of claim 10, wherein the first userinput is received through a first information source, and wherein thesecond user input is received through a second information source. 12.The electronic device of claim 11, wherein the first information sourcecomprises at least one of a voice microphone, a two-dimensional camera,a three-dimensional camera, an infrared camera, a motion sensor, aposture sensor, a wearable device, a shared sensor information source ona remote mobile device, an analog data generator, or an analog remotecontrol, and wherein the second information source comprises at leastone of a mouse, a keyboard, or a remote control.
 13. The electronicdevice of claim 10, wherein the processor is further configured to: setan initial position of a predetermined candidate focus as a positionthat a user recently clicked on the screen, according to a user's clickhistory on the screen.
 14. The electronic device of claim 13, whereinthe plurality of candidate focuses are discretely placed on a pluralityof areas of the screen according to a number of candidate focuses. 15.The electronic device of claim 10, wherein the screen includes akeyboard user interface (UI) including UI elements corresponding to aplurality of characters.
 16. The electronic device of claim 15, whereinthe plurality of candidate focuses are displayed on elementscorresponding to at least two or more characters from among theplurality of characters.
 17. The electronic device of claim 10, whereinthe focus is one of a cursor or a focus box.
 18. The electronic deviceof claim 10, wherein the processor, based on one of the plurality ofcandidate focuses being switched to the active focus, controls thedisplay to display the switched active focus distinctively fromcandidate focuses.
 19. A non-transitory computer-readable recordingmedium having recorded thereon instructions, which when executed by aprocessor, cause an electronic device to perform operations comprising:displaying a plurality of candidate focuses on a screen; switching oneof the plurality of candidate focuses to an active focus according to afirst user input; and moving the switched active focus according to asecond user input.
 20. The non-transitory computer-readable recordingmedium of claim 19, wherein the first user input is received through afirst information source, and wherein the second user input is receivedthrough a second information source.